Method and system of motion estimation with neighbor block pattern for video coding

ABSTRACT

Techniques related to motion estimation with neighbor block pattern for video coding.

BACKGROUND

Due to ever increasing video resolutions, and rising expectations forhigh quality video images, a high demand exists for efficient image datacompression of video while performance is limited for coding withexisting video coding standards such as VP #standards such as VP9. Theaforementioned standards use expanded forms of traditional approaches toaddress the insufficient compression/quality problem, but the resultsare still insufficient.

These video coding processes use inter-prediction at an encoder toreduce temporal (frame-to-frame) redundancy. Motion estimation is a keyoperation in an encoder. Motion estimation is the process of findingareas of a frame being encoded that are most similar to areas of areference frame in order to find the motion vectors. Motion vectors areused to construct predictions for the encoded block. The differencebetween the prediction and real (original) data is called residual dataand is compressed and encoded together with the motion vectors.

By the conventional block-matching, frames are partitioned into blocksof multiple sizes. The individual blocks on a current frame are comparedto some number or each block position of a search window on a referenceframe. The lowest sum of absolute difference (SAD), mean square error(MSE), or other metric is considered a best match. While very accurate,the search reduces performance.

One alternative strategy to increase efficiency is to use the motionvectors already established for neighbor blocks near a current blockbeing predicted. This operates on the principle that blocks near eachother are more likely to have content of the same object in an image,and therefore are more likely to have the same or similar motion vector.Thus, some combination or form of motion vectors of a set of neighborblocks often can be used as candidate motion vectors of the currentblock. For VP9, the neighbor block locations are fixed, pre-determined,and placed on a look-up table for each different block size resulting inhardware that is fixed relative to values on the look-up table andcannot be easily changed when such values are changed. Also, 8×8 pixelpotential neighbor blocks must have both an 8×8 block-based motionvector in the case where the current block is 8×8 or larger, and two 4×4neighbor sub-blocks each to provide a motion vector when adjacent acurrent block that is smaller than an 8×8 block to be predicted.Additionally, the neighbor block pattern extends over a relatively widearea that is three columns to the left and three rows above the currentblock. These limitations result in a large amount of data to be held inmemory during the neighbor block operations increasing required memorycapacity and bandwidth, and therefore required chip area on a memory,and when performed by hardware, such as an accelerator, make itdifficult to implement neighbor block determinations on the hardware byincreasing the required logic, thereby increasing the gate count (orchip area) of the hardware, which also increases costs.

BRIEF DESCRIPTION OF THE DRAWINGS

The material described herein is illustrated by way of example and notby way of limitation in the accompanying figures. For simplicity andclarity of illustration, elements illustrated in the figures are notnecessarily drawn to scale. For example, the dimensions of some elementsmay be exaggerated relative to other elements for clarity. Furthermore,where considered appropriate, reference labels have been repeated amongthe figures to indicate corresponding or analogous elements. In thefigures:

FIG. 1 is an annotated look-up table of neighbor blocks used for motionestimation;

FIG. 1A is a schematic diagram of a partitioned frame to show thenumbering of blocks around a current frame for determining the locationof neighbor blocks;

FIGS. 2A-2M are schematic diagrams showing frame partitioning andneighbor block locations for current blocks of different sizes;

FIG. 3 is an illustrative diagram of an encoder for a video codingsystem;

FIG. 4 is an illustrative diagram of a decoder for a video codingsystem;

FIG. 5 is a flow chart showing a motion estimation process using aneighbor block pattern for video coding according to the implementationsherein;

FIG. 6 is a neighbor block pattern formula used according to theimplementations herein;

FIGS. 7A-7M are schematic diagrams showing frame partitioning andneighbor block locations for current blocks of different sizes and on athree column, one row pattern according to the implementations herein;

FIG. 8 is a schematic diagram showing frame partitioning and neighborblock locations including an external block relative to a current blockaccording to the implementations herein;

FIGS. 9A-9M schematic diagrams showing frame partitioning and neighborblock locations for current blocks of different sizes and on a onecolumn, one row pattern according to the implementations herein;

FIGS. 10A-10B is a detailed flow chart showing a motion estimationprocess using a neighbor block pattern for video coding according to theimplementations herein;

FIG. 11 is an illustrative diagram of an example system in operation forproviding a motion estimation process using a neighbor block pattern forvideo coding according to the implementations herein;

FIG. 12 is an illustrative diagram of an example system;

FIG. 13 is an illustrative diagram of another example system; and

FIG. 14 illustrates another example device, all arranged in accordancewith at least some implementations of the present disclosure.

DETAILED DESCRIPTION

One or more implementations are now described with reference to theenclosed figures. While specific configurations and arrangements arediscussed, it should be understood that this is done for illustrativepurposes only. Persons skilled in the relevant art will recognize thatother configurations and arrangements may be employed without departingfrom the spirit and scope of the description. It will be apparent tothose skilled in the relevant art that techniques and/or arrangementsdescribed herein may also be employed in a variety of other systems andapplications other than what is described herein.

While the following description sets forth various implementations thatmay be manifested in architectures such as system-on-a-chip (SoC)architectures for example, implementation of the techniques and/orarrangements described herein are not restricted to particulararchitectures and/or computing systems and may be implemented by anyarchitecture and/or computing system for similar purposes. For instance,various architectures employing, for example, multiple integratedcircuit (IC) chips and/or packages, and/or various computing devicesand/or consumer electronic (CE) devices such as set top boxes, smartphones, etc., may implement the techniques and/or arrangements describedherein. Furthermore, while the following description may set forthnumerous specific details such as logic implementations, types andinterrelationships of system components, logic partitioning/integrationchoices, etc., claimed subject matter may be practiced without suchspecific details. In other instances, some material such as, forexample, control structures and full software instruction sequences, maynot be shown in detail in order not to obscure the material disclosedherein.

The material disclosed herein may be implemented in hardware, firmware,software, or any combination thereof unless specified herein. Thematerial disclosed herein also may be implemented as instructions storedon a machine-readable medium, which may be read and executed by one ormore processors. A machine-readable medium may include any medium and/ormechanism for storing or transmitting information in a form readable bya machine (e.g., a computing device). For example, a machine-readablemedium may include read only memory (ROM); random access memory (RAM);magnetic disk storage media; optical storage media; flash memorydevices; electrical, optical, acoustical or other forms of propagatedsignals (e.g., carrier waves, infrared signals, digital signals, etc.),and others. In another form, a non-transitory article, such as anon-transitory computer readable medium, may be used with any of theexamples mentioned above or other examples except that it does notinclude a transitory signal per se. It does include those elements otherthan a signal per se that may hold data temporarily in a “transitory”fashion such as RAM and so forth.

References in the specification to “one implementation”, “animplementation”, “an example implementation”, etc., indicate that theimplementation described may include a particular feature, structure, orcharacteristic, but every implementation may not necessarily include theparticular feature, structure, or characteristic. Moreover, such phrasesare not necessarily referring to the same implementation. Furthermore,when a particular feature, structure, or characteristic is described inconnection with an implementation, it is submitted that it is within theknowledge of one skilled in the art to effect such feature, structure,or characteristic in connection with other implementations whether ornot explicitly described herein.

Systems, articles, and methods are described below related to motionestimation with neighbor block pattern for video coding.

As mentioned above, current video coding standards, such as VP9 can beimproved by improving the motion estimation so that speed and efficiencyare increased by reducing the amount of memory transactions, the amountof data that needs to be stored during motion estimation, and byreducing the amount of operations so that the motion estimation, or atleast the neighbor block determination, may be performed with reducedlogic and reduced gate counts on hardware, enabling such operations tobe performed on-chip.

Particularly, during inter-prediction at an encoder, motion estimationis applied to find the best match between an area of a frame such as ablock or sub-block that is being encoded in part of a current frame anda similar block in a reference frame. A motion vector (MV) is thedifference of spatial coordinates of the block being encoded (thecurrent block) and the block in the reference frame being examined. Withthis process, the motion vectors, and the small difference between theblocks just mentioned, are encoded instead of encoding the pixel data ofan entire frame. The motion estimation is applied in a way to find theclosest or best match (or match that is most sufficient) to minimize thecost of the matching process, and strike the right balance betweenprediction accuracy to provide a high quality picture compression andreduction in delay and lags in the streaming or transmission speed ofthe compressed video. The cost is usually computed as a combination of ameasure of the mismatch between the current block and the referenceblock, and the amount of bits used to encode the motion vectors.

For VP9, a checklist list of pre-determined motion vectors from neighborblocks is established. This includes up to nine spatial or neighborblocks on the same frame as the current block being predicted. Onetemporal block is also used and has a motion vector from the previousblock but in the same location as the current block. The motion vectorsof these blocks are selected to form two candidate motion vectors(whether separately or some combined form such a median) to form aprediction list of candidate motion vectors that are provided to formotion compensation. Also, VP9 partitions frames into blocks of varioussizes from 64×64 superblocks down to quad-tree obtained 4×4 blocks. Theselection of the spatial neighbors is complex because each block sizemust have its neighbor block locations determined separately. No genericformula exists that can apply to multiple block sizes.

Referring to FIGS. 1, 1A, and 2A-2M, instead, VP9 provides aconventional pre-determined look-up table such as table 100 where eachrow in the table is provided to correspond to a block of a differentsize, and lists the fixed coordinates of the neighbor blocks that are tobe used for that indicated block size. All coordinates are based on acount of 8×8 blocks. The spatial neighbor locations are retrievedindividually by searching the look-up table 100 by using the currentblock size as the criteria. The table has been annotated to show thecorresponding block size and corresponding partition or blockarrangements 202 to 226 on FIGS. 2A to 2M respectively. Also, thecoordinates on the table are provided as shown on partition example 102(FIG. 1A) where the upper left 8×8 pixel corner block of the currentblock (no matter the size of the current block whether larger than 8×8or small and fitting within the 8×8 block) is considered location (0, 0)and where the up and left directions away from the (0, 0) block locationare negative, and down and to the right are positive. The rows andcolumns are numbered accordingly for clarity. The disclosed method andsystem described below also maintains this numbering scheme.

For example, a 4×4 block listed on the top row of table 100 (andcorresponding to arrangement 202 (FIG. 2A)) has eight neighbor blockseach at a coordinate shown in the row and as located on blockarrangement 202. Since the table provides fixed neighbor block patternsstored at on-chip read-only memory (ROM) for example, the neighbor blockpatterns cannot be easily modified. In order to change the neighborblock coordinates on the table, the hardware must be modified byspecialized manufacturer equipment or access programs, or must bereplaced. This highly limits the ways the hardware can be re-used fornewer neighbor block patterns.

As to the conventional block arrangements 202 to 226, the current blockto be predicted is referred to by its size as (blk a×b) and a slightlydifferent light shading. The numbers shaded darker indicate neighborblock positions 0 to 7 for each block size, and corresponding to theblock coordinates from left to right on each row of the look-up table100. Thus, each block size has eight spatial neighbor blocks.

Referring to FIGS. 2A-2C, block arrangements 202, 204, and 206 havecurrent blocks that are sub-blocks of an 8×8 block and are sizes of 4×4,4×8, and 8×4 respectively. Each of these sub-blocks uses a 4×4 blockmotion vector from sub-blocks 0 and 1 from a top and left neighbor 8×8block and that is adjacent the 8×8 block that the current sub-block iswithin, as shown by block arrangements 202, 204, and 206. Thesesub-blocks also may be used to construct a motion vector predictor listthat holds the motion vectors to be used as candidate motion vectors(explained below), while for all other block sizes, motion vectors of8×8 neighbor blocks are used to construct the motion vector predictorlist. To support these tasks, then, the 8×8 neighbor blocks that couldbe used to provide 4×4 block neighbors for a (4×4), (4×8), or (8×4)current sub-block must have both an 8×8 block (granularity) motionvector stored as well as at least two 4×4 block (granularity) motionvectors stored in a first row and first column buffer for the row andcolumn that is adjacent the 4×4 current sub-block to be predicted. The8×8 blocks must have two 4×4 potential neighbor motion vectors in casethe other 4×4 block location 201 is adjacent a 4×4 current block forexample. Such a configuration undesirably adds memory size requirementsand bandwidth for memory transactions resulting in more delay and morememory area on-chip at both the encoder and decoder, and the complexityof using the sub-blocks as neighbor blocks and current blocks addscomplexity to the hardware logic used to determine the motion vectorswhich also increases gate count (logic area on-chip), line buffer size,and memory bandwidth, which all increases monetary costs.

Also, the VP9 and other standards use a neighbor block pattern extendingover three left columns and three upper rows next to the current blockbeing predicted. Thus, the present systems store all of the motionvectors of the three left columns and three rows as 8×8 block motionvectors (with the 4×4 MVs in the first column and row buffer asmentioned) for each 64×64 superblock, and for both encoding anddecoding. In other words, the length of the data stored for a currentblock is at least as long as the side of a superblock, and needs a threeline buffer for both the columns ad rows. This also increases the memorybandwidth and gate counts since such a large amount of data is to beprocessed for each superblock, which also increases the cost of thesystem.

To resolve these issues, the presently disclosed method and system ofmotion estimation provide a motion vector predictor derivation methodthat reduces memory capacity and bandwidth needs while providing ahardware-operated flexible formula that can be used with a variety ofdifferent block sizes and neighbor block configurations on both theencoder and the decoder. This eliminates the need for a look-up tablethat uses the block size as the search criteria and lists fixed neighborlocations for each block size. The formula is used to identify thespatial neighbor motion vector (MV) locations for multiple or all blocksizes, and the use of 4×4 neighbor block motion vectors has beeneliminated. Thus, now only a single 8×8 motion vector granularity may bestored and used. The present method also reduces memory line buffercapacity requirements from three top rows to one row that may be storedin a single off-chip line buffer, while having at most three leftcolumns that can be stored at on-chip memory. This results in asignificant reduction in required memory capacity and memory bandwidth,and reduces the amount of logic needed to implement the motion predictorderivation hardware, thereby reducing the overall gate count (logicarea) of the hardware.

These features are accomplished by using a neighbor block patternformula that can be used no matter the size of the current block to bepredicted. The neighbor block pattern formula uses the width and/orheight of the current block to determine the location of neighbor blocksrelative to the current block. This includes equations to place theneighbor blocks near corners of the current block or near a center ofthe width or height along an edge of the current block to name a fewexamples. The details are described below.

Now in more detail and while referring to FIG. 3 to place the presentmethods in context, an example video coding system 300 is arranged withat least some implementations of the present disclosure to performmotion estimation with the neighbor block pattern formula describedherein. In various implementations, video coding system 300 may beconfigured to undertake video coding and/or implement video codecsaccording to one or more standards. Further, in various forms, videocoding system 300 may be implemented as part of an image processor,video processor, and/or media processor and undertakes inter-prediction,intra-prediction, predictive coding, and residual prediction. In variousimplementations, system 300 may undertake video compression anddecompression and/or implement video codecs according to one or morestandards or specifications, such as, for example, VP9 or other VP#-based standards or standards that use elements of a VP #standard, butcould also be applied to H.264 (MPEG-4), H.265 (High Efficiency VideoCoding or HEVC), and other standards where relevant or where modified tobe compatible with VP based motion estimation. Although system 300and/or other systems, schemes or processes may be described herein, thepresent disclosure is not necessarily always limited to any particularvideo encoding standard or specification or extensions thereof.

As used herein, the term “coder” may refer to an encoder and/or adecoder. Similarly, as used herein, the term “coding” may refer toencoding via an encoder and/or decoding via a decoder. A coder, encoder,or decoder may have components of both an encoder and decoder.

For the example video coding system 300, the system may be an encoderwhere current video information in the form of data related to asequence of video frames may be received for compression. The system 300may partition each frame into smaller more manageable units (describedin greater detail below), and then compare the frames to compute aprediction. If a difference or residual is determined between anoriginal block and prediction, that resulting residual is transformedand quantized, and then entropy encoded and transmitted in a bitstreamout to decoders or storage. To perform these operations, the system 300may include an input picture buffer (with optional picture reorderer)302, a prediction unit partitioner 304, a subtraction unit 306, aresidual partitioner 308, a transform unit 310, a quantizer 312, anentropy encoder 314, and a rate distortion optimizer (RDO) and/or ratecontroller 316 communicating and/or managing the different units. Thecontroller 316 manages many aspects of encoding including ratedistortion or scene characteristics based locally adaptive selection ofright motion partition sizes, right coding partition size, best choiceof prediction reference types, and best selection of modes as well asmanaging overall bitrate in case bitrate control is enabled.

The output of the quantizer 312 also may be provided to a decoding loop350 provided at the encoder to generate the same reference orreconstructed blocks, frames, or other units as would be generated atthe decoder. Thus, the decoding loop 350 uses inverse quantization andinverse transform units 318 and 320 to reconstruct the frames, andresidual assembler 322, adder 324, and partition unit assembler 326 toreconstruct the units used within each frame. The decoding loop 350 thenprovides filters 328 to increase the quality of the reconstructed imagesto better match the corresponding original frame. This may include adeblocking filter, a sample adaptive offset (SAO) filter, and a qualityrestoration (QR) filter. The decoding loop 350 also may have a decodedpicture buffer 330 to hold reference frames. The encoder 300 also has amotion estimation module or unit 332 that provides candidate motionvectors as referred to below, a motion compensation module 334 that usesthe candidate motion vectors, and an intra-frame prediction module 336.Both the motion compensation module 334 and intra-frame predictionmodule 336 may provide predictions to a prediction modes analyzer andselector 338 that selects the best prediction mode for a particularblock. As shown in FIG. 3, the prediction output of the selector 338 inthe form of a prediction block is then provided both to the subtractionunit 306 to generate a residual, and in the decoding loop to the adder324 to add the prediction to the residual from the inverse transform toreconstruct a frame.

More specifically, the video data in the form of frames of pixel datamay be provided to the input picture buffer 102. The buffer 102 holdsframes in an input video sequence order, and the frames may be retrievedfrom the buffer in the order in which they need to be coded. Forexample, backward reference frames are coded before the frame for whichthey are a reference but are displayed after it. The input picturebuffer also may assign frames a classification such as I-frame(intra-coded), P-frame (inter-coded, predicted from a previous referenceframe), and compound frames (inter-coded frame which is an average oftwo reference frames). In I frames, spatial prediction is used, and inone form, only from data in the frame itself. In P frames, temporal(rather than spatial) prediction may be undertaken by estimating motionbetween frames.

As to the partitioning of the frames, when a VP9 type standard is beingused, the prediction partitioner unit 104 may divide the frames intovery large tiles (such as 3 to a frame) which may or may not be used bythe encoder, and the tiles into superblocks (64×64) as the main dividerof the frame for the encoder. Quad tree derived divisions thereof can beused and as mentioned including horizontal and vertical divisions. Suchdivisions may include block sizes of 32×64, 64×32, 32×32, 32×16, 16×32,16×16, 8×16, 16×8, 8×8, 8×4, 4×8, and 4×4. The 8×8 blocks are the basisof motion vectors during inter-prediction. It should be noted, however,that the foregoing are only example divisions, and many others couldapply including HEVC divisions, such as LCUs, PUs, CUs, etc. As usedherein, the term “block” by itself does not necessarily refer to anyspecific size or shape of block.

Also in video coding system 300, the current video frame divided intopartitions may be provided to the motion estimation unit or estimator332. System 300 may process the current frame in the designated units ofan image in raster or different scan order or in parallel diagonalwave-type processing order where a current block may use the motionvectors already established in blocks above and to the left of thecurrent block. For this purpose, a neighbor block motion vector buffer333 may be provided to hold the motion vectors from previously predictedblocks that are in an area that could be neighbor blocks to a currentblock being analyzed. As described below, the required capacity of thisbuffer 333 has been greatly reduced, and part of the buffer (such as atmost three columns to the left of a current block) may be internal oron-chip, and part of the buffer (such as for a single row or linebuffer) may be external or off-chip. When video coding system 300 isoperated in inter-prediction mode, motion estimation unit 332 maygenerate one or more motion vectors in response to the current videoframe and a reference video frame. A block-based search methodsupplemented by using the motion vectors of neighbor blocks as describedherein may be used to match a block of a current frame with a candidateblock on reference frame, and thereby determine a motion vector to beencoded for a prediction block.

By one example similar to VP9, and as arranged herein, a neighbor blockpattern formula 335 stored in ROM for example, may or may not beoperated by firmware or fixed function hardware, and may be used tocompute spacial neighbor motion vector locations relative to a currentblock on a frame as explained in detail below. The neighbor blocklocations of at most nine (or other number) of spatial neighbor blocksare listed on a checklist. The checklist is reviewed block by block todetermine which blocks have a motion vector. The motion vectors of theneighbor blocks on the checklist then may be placed one by one in an MVpredictor list holding a maximum two candidate MVs to be passed on tothe motion compensator for prediction. Generally, the predictor list isfilled from the checklist and one temporal MV in the order as providedby the formula, and as space becomes available on the predictor list,and by one form, alternating between using the latest spatial MV foundand the temporal MV. More than one reference (such as three)alternatively could be alternatively referred to for the temporal block.

As mentioned, the predictor list may include two MV candidates referredto as the nearest MV which is the first MV on the list, and a near MVwhich is the second MV on the list. A new MV is the nearest MV plus adelta MV derived in a block search. Also, depending on the timing ofwhen an MV is being fetched for prediction at the motion compensatormodule 334, an empty space may be filled with a zero MV (ZMV) whichrefers to no motion from the reference block to the current block MV is(0, 0).

The motion compensation module 334 then may use the reference videoframe and the two candidate motion vectors provided from the predictionlist by motion estimation module 332 to provide multiple candidatemotion vectors to the motion compensator to provide alternativepredictions for selection by the mode selector. Many options arepossible.

The predicted block then may be subtracted at subtractor 306 from thecurrent block, and the resulting residual is provided to the residualcoding partitioner 308. Coding partitioner 308 may partition theresidual into one or more blocks, and by one form for VP9, dividing theprediction partitions further into transform partitions (32×32, 16×16,8×8, or 4×4 for example) which may be division sizes that are not thesame as the prediction (or motion) partition sizes. The relevant blockor unit is transformed into coefficients using variable block sizediscrete cosine transform (VBS DCT) and/or 4×4 discrete sine transform(DST) to name a few examples. Using the quantization parameter (Qp) setby the controller 316, the quantizer 312 then uses lossy resampling orquantization on the coefficients. The generated set of quantizedtransform coefficients may be reordered and entropy coded by entropycoding module 314 to generate a portion of a compressed bitstream (forexample, a Network Abstraction Layer (NAL) bitstream) provided by videocoding system 300. In various implementations, a bitstream provided byvideo coding system 300 may include entropy-encoded coefficients inaddition to side information used to decode each block (e.g., predictionmodes, quantization parameters, motion vector information, partitioninformation, in-loop filtering information, and so forth), and may beprovided to other systems and/or devices as described herein fortransmission or storage.

The output of the quantization module 312 also may be provided tode-quantization unit 318 and inverse transform module 320 for thedecoding loop 350. De-quantization unit 318 and inverse transform module320 may implement the inverse of the operations undertaken by transformunit 310 and quantization module 312. A residual assembler unit 322 thenmay reconstruct the residuals. The output of the residual assembler unit322 then may be combined at adder 324 with the predicted frame togenerate a rough reconstructed block. A partition unit assembler 326then reconstructs the partitions to complete the frame reconstruction.

The quality of the reconstructed frame is then improved by running theframe through the filters 328. The filtered frames are provided to adecoded picture buffer 330 where the frames may be used as referenceframes to construct corresponding predictions for motion estimation andcompensation as explained above and as described below. When videocoding system 300 is operated in intra-prediction mode, intra-frameprediction module 336 may use the reconstructed pixels of the currentframe to undertake intra-prediction schemes that will not to bedescribed in greater detail herein.

In some examples, video coding system 300 may include additional itemsthat have not been shown in FIG. 3 for the sake of clarity. For example,video coding system 300 may include a processor, a radio frequency-type(RF) transceiver, splitter and/or multiplexor, a display, and/or anantenna. Further, video coding system 300 may include additional itemssuch as a speaker, a microphone, an accelerometer, memory, a router,network interface logic, and so forth.

Referring to FIG. 4, a system 400 may have, or may be, a decoder, andmay receive coded video data in the form of bitstream 402. The system400 may process the bitstream with an entropy decoding module 404 toextract quantized residual coefficients as well as the motion vectors,prediction modes, partitions, quantization parameters, filterinformation, and so forth. The system 400 then may use an inversequantization module 404 and inverse transform module 406 to reconstructthe residual pixel data. The system 400 then may use a residual codingassembler 408, an adder 410 to add the residual to the predicted block,and a partition unit assembler 412. The system 400 also may decode theresulting data using a decoding loop employing, depending on the codingmode indicated in syntax of bitstream 402 and implemented via predictionmode switch or selector (which also may be referred to as a syntaxcontrol module) 422, either a first path including an intra predictionmodule 420 or a second inter-prediction decoding path including one ormore filters 414. The second path may have a decoded picture buffer 416to store the reconstructed and filtered frames for use as referenceframes as well as to send off the reconstructed frames for display orstorage for later viewing or another application or device. A motioncompensated predictor 418 utilizes reconstructed frames from the decodedpicture buffer 416 as well as motion vectors from the bitstream andstored in a motion vector buffer 417 including candidate neighbor blockmotion vectors to reconstruct a predicted block.

Alternatively, the decoder 400 could have motion estimation capabilityand also has its own motion estimation unit 413. In this case, themotion estimation unit 413 may have a neighbor block motion vectorbuffer 415 to store motion vectors of potential neighbor blocks in anarea that can be used as neighbor blocks (such as the three columns andone row), and may have part on-chip and part external memory asmentioned above for the encoder. Also as mentioned for the encoder, themotion estimator 413 at the encoder has or has access to a neighborblock pattern formula 419 stored in ROM as with the encoder as well. Inthis case, the decoder also determines neighbor block locations for acurrent block, lists the located blocks on a checklist, determines whichblocks on the checklist have a motion vector, and places two of thefound motion vectors onto the prediction list of candidate motionvectors. The motion compensated predictor 418 then uses the candidatemotion vectors to form predictions.

A prediction modes selector 422 uses the predictions and sets thecorrect mode for each block. The functionality of modules describedherein for systems 300 and 400, except for the motion estimation unit332 and 413 described in detail below, are well recognized in the artand will not be described in any greater detail herein.

Referring now to FIG. 5, an example process 500 is arranged inaccordance with at least some implementations of the present disclosure.In general, process 500 may provide a computer-implemented method ofmotion estimation with neighbor block pattern for video coding. In theillustrated implementation, process 500 may include one or moreoperations, functions or actions as illustrated by one or more ofoperations 502 to 508 numbered evenly. By way of non-limiting example,process 500 will be described herein with reference to operationsdiscussed with respect to FIGS. 3-4 above and may be discussed withregard to example systems 300, 400 or 1200 discussed below.

The process 500 may comprise “receive multiple frames of pixel data”502, and particularly at a motion estimation unit within a decoding loopon an encoder that receives reconstructed and filtered reference framesfrom buffer 330 as well as data of current frames to be encoded. Thisalso includes the data defining the partitions or blocks on each frame.

The process 500 also may comprise “determine at least one motion vectorcandidate extending between a current block on a current frame and areference block on a reference frame” 504. Thus, duringinter-prediction, motion estimation involves determining motion vectorsextending between a current block on a current frame and a matchingreference block on a reference frame by block matching techniques. Inorder to reduce the amount of block matching due to its heavycomputational loads and time consumption, motion vectors from blockspreviously analyzed may be used to determine the motion vectors of thecurrent block. By one form, this may include determining a group ofmotion vectors from neighbor blocks relative to the current block. Whenany one of these neighbor motion vectors are placed on the currentblock's prediction list, the motion vectors become candidate motionvectors that may be provided to the motion compensation module todetermine a prediction for the current block as already explainedherein.

The process 500 then may comprise “determine spatial neighbor blocklocations on the same frame as the current block using a neighbor blockpattern formula to compute at least one neighbor block coordinaterelative to the position of the current block on the current frame andusing substantially the same neighbor block pattern formula regardlessof the size of the current block” 506. Thus, instead of a look-up tablethat uses the current block size as the search criteria and lists fixedcoordinate values for each current block size, a neighbor block patternformula is provided that has an equation to compute one or morecoordinates for a neighbor block, where there is at least one equationto compute a coordinate in the formula. By one example form, describedbelow, the formula provides coordinates for neighbor blocks 0 to 8 whereseven of the neighbor blocks have one coordinate that is calculated byan equation. By one example, the calculated coordinate is based on thewidth or height of the current block as counted in the number of 8×8blocks within the current block. By another form, the equations alignthe neighbor block with either a corner or edge of the current block, orwith a center of a side of the current block. This significantlyincreases the flexibility of fixed function hardware or firmwareoperating the formula. Other examples are possible, and the details areprovided below.

The process 500 also may comprise “use motion vectors of the neighborblocks at locations determined by using the at least one neighbor blockcoordinate and to determine at least one motion vector of the currentblock” 508. As explained for a VP #type coding standard, once thecoordinates of a neighbor block is determined, the neighbor blocklocation may be placed on a checklist. The block locations on thechecklist are reviewed one by one until a block with a motion vector ofthat neighbor block is found and obtained from memory, such as RAM orcache. The neighbor block motion vector then may be placed in order asobtained on a prediction list when a space opens on the prediction listand that holds two candidate motion vectors that may include thenearest, near, new, and/or ZMV motion vectors for the current block forexample. These two candidate motion vectors may be provided to themotion compensation unit for determining prediction data of the currentblock. It will be understood that different video coding standards couldbe used as well. Also as mentioned, once a prediction is determined, itis provided to a mode selector, and if selected, it is differenced fromthe actual block and a residual is coded and transmitted to a decoderalong with the motion vector. Thereafter, the decoder may reconstructthe frames either by using a motion estimator as well that findsneighbor blocks and provides their neighbor motion vectors to aprediction list, or by transmitting and using the candidate motionvectors and residuals to reconstruct the frames instead, and so thatsubsequently the frame may be displayed or stored. Thus, it can bestated that the neighbor motion vector pattern formulas may be used toencode or decode the frame or both and then to display the frame.

Some of the other features explained herein include the use of onlystoring 8×8 block-based motion vectors even for sub-blocks smaller than8×8, and a limited neighbor block area for searching for placingneighbor blocks of at most three columns to the left of the currentblock, and one top row for one line buffer where the three columns canbe placed in on-chip memory while the one line buffer is placed onexternal memory (where external is relative to a chip or die, such as anSoC.

Referring now to FIG. 6, one example neighbor block pattern formula 600,as with formulas 335 and 419, lists the coordinates or equations todetermine the coordinates for neighbor blocks 0 to 8, and both ahorizontal and vertical coordinate for each neighbor block. Thecoordinate system remains as explained with system 102 (FIG. 1A) wherethe upper left 8×8 block on the current block is the (0, 0) point of thecoordinate system. The formula 600 is shown in pseudo code form. Thepositioning of each neighbor blocks 0 to 8 is shown on each block orpartition arrangements 702 to 726 (FIGS. 7A to 7M).

The example formula 600 first includes two constants to be used in theequations including the width of the current block (num_8×8_blocks_wide)which is a count of the 8×8 blocks along a side of the current block, orthe number of pixels (bw) on the side divided by eight (shown as a bitshift >>3 here). The other constant is similar except in the height or ydirection to establish the height of the current block(num_8×8_blocks_high). Next, a line on the formula 600 may be providedto determine the coordinate in both x and y directions for each neighborblock 0 to 8. When the side of the current block is less than one block,such as with the 4×4, 4×8, or 8×4 sub-blocks, the width and/or heightthat is the short side is set at 1.

Starting with neighbor block 0, height coordinate (or row number) is thecurrent block height−1, and the width coordinate (or column number) is−1. This places the 0 neighbor block next to the bottom left corner ofthe current block along the left edge of the current block and alignedwith the bottom edge of the current block (shown clearest at block 0 onarrangement 726 (FIG. 7M) and is referred to as the “left” position.

For neighbor block 1, the row coordinate is −1 and the column coordinateis the equation current block width−1, which positions the 1 neighborblock next to the top right corner of the current block along the topedge of the current block and aligned with the right edge of the currentblock (see block 1 on arrangement 726 (FIG. 7M)). This position isreferred to as the “top” position. For neighbor block 2, the rowcoordinate is −1 and the column coordinate is the equation is (currentblock width−1) divided by 2 (shown as bit shift >>1), which positionsthe 2 neighbor block in the center of the width of the current block andalong the top edge of the current block (see block 2 on arrangement 726(FIG. 7M)). This position is referred to as the “top-center” position.

For neighbor block 3, the row coordinate is the equation (current blockheight−1) divided by 2 similar to block 2 except with the height, andthe column coordinate is −1. This positions the 3 neighbor block in thecenter of the left side of the current block and along the left edge ofthe current block (see block 3 on arrangement 726 (FIG. 7M)). Thisposition is referred to as the “left-center” position.

For neighbor block 4, the coordinates are (−1, −1) which touches, and isdiagonal to, the upper left corner of the (0, 0) block in the currentblock, and is referred to as the top-left position (see block 4 onarrangement 726 (FIG. 7M)).

For neighbor block 5, the row coordinate is −1, and the columncoordinate is equal to the current block width (num_8×8_blocks_wide).This positions the 5 neighbor block diagonal to the upper right cornerof the current block and along the top edge of the current block. Thisis considered an external position since it is outside of the column(s)holding the current block, and is referred to as the “top-right”position (see block 5 on arrangement 726 (FIG. 7M)).

For neighbor block 6, the column coordinate is −1, and the rowcoordinate is equal to the current block height (num_8×8_blocks_high).This positions the 6 neighbor block diagonal to the bottom left cornerof the current block and along the left edge of the current block. Thisalso is considered an external position since it is outside of therow(s) holding the current block, and is referred to as the“left-bottom” position (see block 6 on arrangement 726 (FIG. 7M)).

For the neighbor block 7, the row coordinate is −1, and the columncoordinate −3. This is referred to as thetop-left-in-the-third-neighbor-column (or far left-top) position (seeblock 7 on arrangement 726 (FIG. 7M)).

For the neighbor block 8, the row coordinate is (num_8×8_blocks_high−1),and the column coordinate is −3. This position is referred to asleft-in-the-third-neighbor-column (or just far left) (see block onarrangement 726 (FIG. 7M)).

The order of the neighbor blocks in the formula is maintained as aspatial neighbor checklist once the motion vectors are obtained, and areprovided to the prediction list in that order (unless neighbor motionvectors are combined as explained below) which therefore is first-in,first-out (FIFO). The selected order for choosing MVs from the checklistto be placed in the prediction list is used by both the encoder and thedecoder. The selection of MVs from the checklist for the prediction listis performed by using known criteria and is explained in greater detailbelow with process 1000.

Thus, the order of the neighbor block locations in the formula is setwith the most important first to the least important last since theneighbor blocks, and their MVs, are selected in FIFO order. It will beunderstood that the order of the neighbor block positions 0 to 8 withinthe formula could be changed as well when other orders are desired.

Also, the formula 600 is merely one example and many examples can beused that are different than this example including those that add orremove any of the neighbor block positions listed in the formula 600, oruse different positions. By one alternative example, only neighborblocks 0 to 7 are used. By another example, far left positions 7 and 8are removed and more central positions are added referred to as left/2(or L/2) and top/2 (or T/2), where for a current block of 8×8,L/2=8/2=row 4, and T/2=8/2=column 4 column as shown on arrangement 726.Many other variations are possible.

Referring to FIGS. 7A-7M, a neighbor block arrangement or pattern isprovided for each available block size in a standard similar to VP9 forone example, and using the formula 600 to place the neighbor blocks asnumbered on each arrangement 702 to 726 where the current block and itssize for the arrangement is designated as blka×b (e.g., blk4×4). Whereneighbor block positions are missing, the later neighbor blocks areremoved as duplicate to another earlier neighbor block already on thearrangement. For example, referring to arrangement 712 (FIG. 7F), theneighbor block 3 is missing because the coordinates of this neighborblock is (0, −1) because the current block height is 1 which means thatthe row location is row 0 ((1−1)/2=0 according to the formula) since thecurrent block is 16×8. Neighbor block 0 already has coordinates (0, −1)so the neighbor block 3 is not used and is discarded if it is computedat all.

It also will be noticed that all of the arrangements 702 to 726 remainwithin an area that is one row above the current block, and at mostthree columns immediately to the left of the current block. It could beone or the other but here both limitations are used. This reduction inthe pattern space significantly reduces the memory access for neighborblocks since all possible spatial neighbors are stored for the patternspace when determining the neighbor block locations. This patterneliminates two full top rows. This reduction in required memory capacitypermits the at most three columns to be placed on on-chip memory. Thismay increase the on-chip area (or gate count) but it does notsignificantly affect the memory bandwidth negatively. Usually,increasing the on-chip memory is preferable to increasing the line ormotion vector buffer memory. However, the one top row line buffer isstill stored off-chip or at external memory to be able to accommodatelarge frame widths such as with ultra HD clips for 8K video for example.The buffer line may store the MVs of the blocks of the same width as thecurrent block plus one external block (position 5 on the formula).Reducing the area from three rows to a single row buffer, however, stillsignificantly reduces the memory size and bandwidth requirements. Itwill be understood that the formula may be modified to be used withother alternative neighbor block areas instead such as at most (1, 2, or3) columns and 3 row area, or other reduced areas such as 1 column and 3above rows, as long as a current block still will have sufficientpreviously analyzed blocks with motion vectors above and to the left ofthe current block.

Referring to FIGS. 7A-7C, the sub-block arrangements 702 to 706 showthat the current blocks of sub-block size (4×4, 8×4, 4×8) all haveneighbor blocks that are 8×8. As mentioned, this significantly reducesthe amount of data stored by eliminating the two 4×4 block motionvectors for each 8×8 block that was stored previously.

Referring to FIG. 8, a block arrangement or pattern 800 is shown for a32×32 current block 804 that forms the upper right quadrant of a 64×64superblock 802 so that the neighbor block 5 location determined for thecurrent block 504 is external to the column and row of the superblock804 and to the right of the superblock (it is not above or to the leftof the superblock). In this case, the neighbor block 5 (or thislocation) is deemed unavailable or unusable and is not checked when thisneighbor block is listed on the checklist and exceeds the currentsuperblock column as shown. This is performed to maintain the angled(such as 45 degree) wave-front parallelism used to continuously providepreviously analyzed blocks above and to the left of the current block.

Referring to FIGS. 9A-9M, another possible alternative formula isdemonstrated where the formula, and in turn the check list, merelyincludes neighbor blocks locations 0 to 6 (as defined in formula 600)remaining within a neighbor block area that is merely one row and onecolumn both adjacent the current block. Arrangements 902 to 926demonstrate how this would apply to the various block sizes and wherethe current block is designated blka×b on each arrangement. This optionalso includes the feature from above that full 8×8 blocks are used asneighbor blocks to the sub-blocks (FIGS. 9A-9C) so that the 8×8block-based motion vectors are placed on the checklist for thesub-blocks. With this configuration, only the first row and column needto be stored to determine neighbor block locations and motion vectorsfor a current block. This significantly reduces the required memorycapacity and bandwidth, as well as chip size (gate count) as explainedabove.

Referring now to FIGS. 10A-10B, a detailed example motion estimationprocess 1000 is arranged in accordance with at least someimplementations of the present disclosure. In general, process 1000 mayprovide another computer-implemented method of motion estimation withneighbor block pattern formula for video coding. In the illustratedimplementation, process 1000 may include one or more operations,functions or actions as illustrated by one or more of operations 1002 to1036 numbered evenly. By way of non-limiting example, process 1000 willbe described herein with reference to operations discussed with respectto FIGS. 3-9M and 12, and may be discussed with reference to examplesystems 300, 400, and/or 1200 discussed below as well as neighbor blockpattern formula 600.

Process 1000 may include “obtain image data of frame” 1002, andparticularly, obtaining chroma and luminance pixel data of frames to beencoded as well as reconstructed reference frames as described above.This also may include obtaining data that indicates the partition (orblock) locations on each frame. Herein, the terms partition and blockare used interchangeably.

Process 1000 may include “determine motion vectors of blocks atupper-left corner first row(s) and first column(s) of frame” 1004. Thus,for systems that use wave-front parallel processing for neighbor blockmotion vector use, the process may still start with block searching fornew motion vector derivation at least along the left and top edges ofthe frame. The process may often start by using ZMV on the first blockin the upper left corner of the frame, and then either performing newblock matching searches on the adjacent left and lower blocks, or byusing the ZMV of the first block or both. This process forms a parallel,angled wave-front of blocks that may use the previously analyzed upperand left blocks to determine the motion vector and other data of thecurrent block on the wave-front. Once a sufficient amount of blocks havebeen analyzed and motion vectors have been determined, such as whenthree left columns and one top row are processed and available asneighbor blocks, then the process can switch to neighbor block motionvector derivation.

Process 1000 may include “set partition p=0 for first partition and MAXP for last partition” 1006. This operation starts a partition or blockcounter for the process and may save the maximum number of partitions onthe frame to be analyzed for neighbor block MVs. Other alternatives arepossible.

Process 1000 may include “buffer, in whole or in part, at most threeleft columns and one above row relative to the partition p” 1008, andparticularly place the available or permissible neighbor block patternarea of a partition in memory where it is accessible to retrieve MVs ofneighbor blocks. As described above, this area may include three leftcolumns and one top row adjacent the current block (or partition) to beanalyzed. The area is reduced in order to reduce required memorycapacity and bandwidth so that the buffer for the at most three leftcolumns may even be placed on-chip, although the single line buffer forthe single top row is still placed on external memory. This also mayreduce memory hardware area as explained above. By another alternative,only one row and one column adjacent the current block is used as thearea, and other alternatives may exist as well.

Process 1000 may include “use neighbor block pattern formula regardlessof current block size to determine neighbor block locations relative topartition p” 1010. Specifically, a neighbor block pattern formula, suchas formula 600 by one example, may be provided and that lists thecoordinate or equation for calculating the coordinate for each neighborblock location listed in the formula. The formula may list the neighborlocations in a certain order and that is to be provided in a checklistand is checked in that order for placement onto a prediction list. Theformula may include the same list of neighbor block locations as informula 600 or any combination of them or something having completelydifferent locations as long as at least one coordinate of at least oneneighbor block location is computed by using an equation in the formulaand the equation applies to multiple partition (or current block) sizes.Example formula 600 uses an equation for seven coordinates.

Thus, this operation 1010 may include “set partition width(num_8×8_blocks_wide) and partition height (num_8×8_blocks_high) basedon number of 8×8 blocks in p” 1012. By one example, the equations in theformula for computing coordinates are based on the width or height ofthe partition, although it could be some other dimension (such as area).The width and height are measured in the number of 8×8 blocks (or numberof pixels divided by 8). This operation also may include “set widthand/or height bw or bh<8 to 1” 1014, which refers to using the width andheight of 1 for a side of each of the sub-blocks (4×4, 4×8, or 8×4) thatis less than 8 pixels. This permits full 8×8 block positions to be usedas neighbor blocks for the sub-blocks so that only 8×8 block-based MVsneed be stored in memory significantly reducing the amount of memory andhardware required to determine neighbor block MVs as described above.

Process 1000 may include “compute neighbor block coordinates usingpartition width and/or partition height” 1016. Also as described abovewith formula 600, the width and height may be used in equations to placethe neighbor block by a corner of the partition (current block), whichprovides an equation such as:

((width or height)−1)  (1)

which is for neighbor block 0 or 1 on formula 600 for example, or at acenter along the left edge or top edge of the partition with:

((width or height)−1)/2  (2)

which is neighbor block 2 or 3 on formula 600 for example, or a positionexternal to the column and row of the partition (current block) which isthe value of the width or height itself (such as neighbor blocks 5 or 6on formula 600). It will be understood, however, that many otherpositions could be determined as well by using the width or height ofthe current block or by using another dimension.

Process 1000 may include “eliminate duplicate locations” 1018, and asmentioned for smaller current blocks, two of the neighbor blocklocations determined from the formula may be the same location on theframe. By one form, the later neighbor block is dropped from thechecklist or ignored.

Optionally, process 1000 may include “eliminate locations outside ofwave-front parallel permissible locations” 1020, and as explained witharrangement 800, blocks that are external to the superblock of thecurrent partition are marked as unavailable and may be dropped from thechecklist so that the computations for the parallel, angled wave-frontremains consistent and does not include blocks that are more likely toinject inaccuracies for example.

Process 1000 may include “place neighbor block locations on a checklist”1022, and therefore, once one of the 8×8 neighbor block locations is setby using the formula, the 8×8 location is added to the checklist.Thereafter, the checklist is checked to determine which neighbor blocklocations have a motion vector and are therefore available to be placedon the prediction list. Thus, process 1000 may include “determine whichneighbor block locations on the checklist have motion vectors” 1024.This recognizes that in some cases some of the locations may not have amotion vector, such as those that were a skip location or were intracoded instead for example. Once it is determined which neighbor blocklocations are available (have motion vectors), the process places two ofthe motion vectors onto the prediction list. The neighbor blocklocations are checked in the same order as set in the formula, and ischecked in an FIFO manner. If a location does not have a motion vector,it is discarded and the next location is analyzed.

It will be understood that in addition to the singular neighbor blockmotion vectors, the neighbor block motion vectors may be combined beforebeing placed in the checklist, or before being placed in the predictionlist. Thus, by one form, a partition may have the median motion vectorof its neighbor blocks placed in the checklist. Many other alternativesare possible.

Process 1000 may include “select motion vector(s) as candidate motionvectors on prediction list for partition p” 1026. For this operation,and similar to the VP9 standards, the motion vector (MV) predictor listmay include two MVs, and in one form, only two MVs. The two MVs arechosen from the nine spatial neighbor MVs obtained from the neighborblocks as just explained, and one temporal MV. The predictor list isfilled by the following process:

-   -   a. First, a check of the nine spatial MVs (extending from the        same reference picture) is performed one by one and an available        MV is inserted to the MV predictor list. If the MV predictor        list is full, then the check is stopped.    -   b. If the MV predictor list is not full, then check the        collocated MV from a previous frame using the same reference        frame.    -   c. If the MV predictor list is still not full, check the nine        spatial MVs (of a different reference picture) and one by one.    -   d. If the MV predictor list is still not full, check the        collocated MV from the previous frame and with the different        reference.    -   e. If the MV predictor list is still not full, fill the vacancy        with zero (0, 0) MVs.

In the VP9 codec, the first MV in the MV predictor list is called thenearest MV, and the second MV in the predictor list is the near MV. Anew MV is equal to the nearest MV plus a delta MV, where the delta MV isderived from a block matching search.

It will be appreciated that other standards and other algorithms forusing the neighbor block motion vectors could be used here, such as withHEVC where alternative partitioning is provided for a single frame andmay be compared by using inter-prediction. Other examples algorithms andmethods may be used with other video coding standards as well.

Process 1000 may include “provide candidate motion vectors for motioncompensation and mode selection” 1028, and once a motion vector is acandidate motion vector on the prediction list, it may be provided tothe motion compensator to determine an alternative prediction for eachmotion vector. Each prediction is then provided to the mode selector toselect the best prediction based on accuracy and/or cost for coding theresidual of a prediction. The winning prediction is then subtracted fromthe original data of the frame to generate a residual, and the residualas well as the corresponding motion vector is coded and transmitted to adecoder. By one form, all candidate motion vectors and their predictionsare transmitted to the decoder so that a selector at the decoder canalso select among the candidate motion vectors.

Process 1000 then may include query “p=MAX P?” 1030 to determine whetherthe last partition has been reached. If not, the process 1000 mayinclude “set p=p+1” 1032, to tick the partitioner counter up one, andthen may obtain 1034 the next partition p. The process repeats until thelast partition Max P is obtained and analyzed. Then, process 1000 mayinclude “repeat for individual frames until end of frame sequence” 1036.Once the last frame is analyzed, the process ends.

Referring now to FIG. 11, system 1200 may be used for an example process1100 of motion estimation with neighbor block pattern formula for videocoding shown in operation, and arranged in accordance with at least someimplementations of the present disclosure. In the illustratedimplementation, process 1100 may include one or more operations,functions, or actions as illustrated by one or more of actions 1102 to1132 numbered evenly, and used alternatively or in any combination. Byway of non-limiting example, process 1100 will be described herein withreference to operations discussed with respect to any of theimplementations described herein.

In the illustrated implementation, system 1200 may include a processingunit 1220 with logic units or logic circuitry or modules 1250, the like,and/or combinations thereof. For one example, logic circuitry or modules1250 may include the video encoder 300 with a motion estimation unit1252, and optionally a decoder 400 with an estimation unit 1254, thatperforms many of the operations related to the neighbor block patternformula 1208 and determination of neighbor block locations as mentionedabove, and optionally the video decoder 400. Although system 1200, asshown in FIG. 12, may include one particular set of operations oractions associated with particular modules, these operations or actionsmay be associated with different modules than the particular moduleillustrated here.

Process 1100 may include “obtain video data of original andreconstructed frames” 1102, where the system, or specifically a motionestimation unit at the encoder, may obtain access to pixel data ofreconstructed frames. The data may be obtained or read from RAM or ROM,or from another permanent or temporary memory, memory drive, or libraryas described on systems 1200 or 1300, or otherwise from an image capturedevice. The access may be continuous access for analysis of an ongoingvideo stream for example. Process 1100 then may include “obtain currentframe and reference frame data including partition data” 1104 of areconstructed frame so that blocks to be encoded can be matched toreference blocks during the motion estimation search.

Process 1100 may include “buffer at most three left columns or threerows, and one opposite row or column relative to the current block”1106, and as mentioned above, to have all or most of the motion vectorsthat might be used as neighbor block MVs. As noted, at most threecolumns may be used to provide the column buffer on on-chip memory. Alsoas mentioned, this is one alternative area and others could be used aswell such as that shown for arrangements 902 to 926 (FIGS. 9A-9M) withone row and one column.

Process 1100 may include “obtain block width and height” 1108, and alsoas mentioned above, computed as the number of 8×8 blocks on a side of acurrent block.

Process 1000 may include “use neighbor block pattern formula to computecoordinates of neighbor blocks to determine neighbor block locationsrelative to current block and by using the same formula regardless ofcurrent block size and by using the current block width or height orboth” 1110. Thus, the equations use the width or height of the currentblock as a variable in the equation to determine a coordinate for aneighbor block location. In this way, the formula is not limited to anyparticular block size, and by one form, none of the neighbor blocklocations are limited by block size. The details are explained above.

Process 1100 may include “compute neighbor blocks for 8×8 sub-blocks byusing 1 as the sub-block width or height or both for the width or heightof the current block that is less than 8 pixels” 1112, and as mentionedabove, the sub-blocks now use full 8×8 neighbor blocks so that 4×4block-based MVs no longer need to be stored to reduce memory andhardware requirements as explained above.

Process 1100 may include “select neighbor blocks on at most three leftcolumns or three rows, and one opposite row or column relative to thecurrent block” 1114, and again, the permissible area for neighbor blockis reduced from conventional areas to reduce memory and hardware needsas explained above. Another option is provided as one row and onecolumn, but other area configurations could also be used.

Process 1100 may include “use motion vectors of located neighbor blocksto determine candidate motion vectors to be used by motion compensatorto provide prediction(s) to mode selector for current block” 1116. Asmentioned above, with VP9, two of the nine motion vectors on a checklistfor a current block and one block that is a temporal block are chosen tobe placed on the prediction list of candidate motion vectors to beprovided to the motion compensator as already explained above.

Process 1100 may include repeating this part of the process for a nextpartition of the frame if any exists 1117, until all or multiplepartitions have been analyzed, and this may be repeated for each framein a video sequence that is to be analyzed by inter-prediction.

Process 1100 then may continue and include “determine residuals based onmode selection, generate and transmit bitstream with encoded data” 1118,including transmission of frame data, residual data, and motion vectordata including the neighbor motion vectors that were selected ascandidates and placed on the prediction list.

The decoder 200 then may be provided to “decode frame data, residuals,and motion vectors” 1120, which by one form, either includes pre-storingthe neighbor block pattern formula at the decoder or transmitting theformula with the image data to the decoder so that the decoder canperform candidate motion vector generation from a generated neighborblock motion vector checklist. In other words, the process 1100 mayrepeat 1122 operations 1108 to 1117 as shown to generate the candidatemotion vectors at the decoder and by using the stored neighbor blockpattern formula. Accordingly, this may include using the formula todetermine neighbor block locations, placing the locations on thechecklist, checking the locations on the checklist to determine whichlocations have motion vectors, and then placing the neighbor blockmotion vectors on a prediction list using the order of the formula orother set order, and when a space becomes available on the predictionlist as with the encoder prediction list.

The next operation then is “use motion compensation to constructprediction blocks by using the motion vectors” 1124, and “add theresiduals to the prediction blocks to form reconstructed blocks” 1126.Process 1100 then may continue with “use reconstructed frames asreference frames for the motion compensation” 1128, and “repeat formultiple frames until the end of the sequence” 1130. The reconstructedframes also may be provided for display and/or storage 1132.

In general, logic units or logic modules, such as that used by encoder300 and decoder 400 may be implemented, at least in part, by hardware,software, firmware, or any combination thereof. As shown, in someimplementations, encoder and decoder 300/400 may be implemented viaprocessor(s) 1203. In other implementations, the coders 300/400 may beimplemented via hardware or software implemented via one or more othercentral processing unit(s). In general, coders 300/400 and/or theoperations discussed herein may be enabled at a system level. Someparts, however, for enabling the motion estimation in an encoding loop,and/or otherwise controlling the type of compression scheme orcompression ratio used, may be provided or adjusted at a user level, forexample.

It will be appreciated that this neighbor block usage may be provided ona system that uses alternative search strategies where this strategy isonly one option used, or where a group of different motion estimationprocesses are used and the one with the best result is ultimately usedfor encoding, or where the results from a number of the search processesare combined, such as a mean or median, and then the combination resultis used. This may include direct methods such as block-based searcheswith alternative search pattern arrangements for example, and/or phasecorrelation, frequency domain, pixel recursive, and/or opticalflow-based algorithms, and/or indirect methods such as corner detection,object tracking and other statistical function based algorithms.

While implementation of example process 500, 1000, and/or 1100 mayinclude the undertaking of all operations shown in the orderillustrated, the present disclosure is not limited in this regard and,in various examples, implementation of any of the processes herein mayinclude the undertaking of only a subset of the operations shown and/orin a different order than illustrated.

In implementations, features described herein may be undertaken inresponse to instructions provided by one or more computer programproducts. Such program products may include signal bearing mediaproviding instructions that, when executed by, for example, a processor,may provide the functionality described herein. The computer programproducts may be provided in any form of one or more machine-readablemedia. Thus, for example, a processor including one or more processorcore(s) may undertake one or more features described herein in responseto program code and/or instructions or instruction sets conveyed to theprocessor by one or more machine-readable media. In general, amachine-readable medium may convey software in the form of program codeand/or instructions or instruction sets that may cause any of thedevices and/or systems described herein to implement at least portionsof the features described herein. As mentioned previously, in anotherform, a non-transitory article, such as a non-transitory computerreadable medium, may be used with any of the examples mentioned above orother examples except that it does not include a transitory signal perse. It does include those elements other than a signal per se that mayhold data temporarily in a “transitory” fashion such as RAM and soforth.

As used in any implementation described herein, the term “module” refersto any combination of software logic, firmware logic and/or hardwarelogic configured to provide the functionality described herein. Thesoftware may be embodied as a software package, code and/or instructionset or instructions, and “hardware”, as used in any implementationdescribed herein, may include, for example, singly or in anycombination, hardwired circuitry, programmable circuitry, state machinecircuitry, and/or firmware that stores instructions executed byprogrammable circuitry. The modules may, collectively or individually,be embodied as circuitry that forms part of a larger system, forexample, an integrated circuit (IC), system on-chip (SoC), and so forth.For example, a module may be embodied in logic circuitry for theimplementation via software, firmware, or hardware of the coding systemsdiscussed herein.

As used in any implementation described herein, the term “logic unit”refers to any combination of firmware logic and/or hardware logicconfigured to provide the functionality described herein. The“hardware”, as used in any implementation described herein, may include,for example, singly or in any combination, hardwired circuitry,programmable circuitry, state machine circuitry, and/or firmware thatstores instructions executed by programmable circuitry. The logic unitsmay, collectively or individually, be embodied as circuitry that formspart of a larger system, for example, an integrated circuit (IC), systemon-chip (SoC), and so forth. For example, a logic unit may be embodiedin logic circuitry for the implementation firmware or hardware of thecoding systems discussed herein. One of ordinary skill in the art willappreciate that operations performed by hardware and/or firmware mayalternatively be implemented via software, which may be embodied as asoftware package, code and/or instruction set or instructions, and alsoappreciate that logic unit may also utilize a portion of software toimplement its functionality.

As used in any implementation described herein, the term “component” mayrefer to a module or to a logic unit, as these terms are describedabove. Accordingly, the term “component” may refer to any combination ofsoftware logic, firmware logic, and/or hardware logic configured toprovide the functionality described herein. For example, one of ordinaryskill in the art will appreciate that operations performed by hardwareand/or firmware may alternatively be implemented via a software module,which may be embodied as a software package, code and/or instructionset, and also appreciate that a logic unit may also utilize a portion ofsoftware to implement its functionality.

Referring to FIG. 12, an example video coding system 1200 for providingmotion estimation with a neighbor block pattern formula for video codingmay be arranged in accordance with at least some implementations of thepresent disclosure. In the illustrated implementation, system 1200 mayinclude one or more central processing units or processors 1203, adisplay device 1205, and one or more memory stores 1204. Centralprocessing units 1203, memory store 1204, and/or display device 1205 maybe capable of communication with one another, via, for example, a bus,wires, or other access. In various implementations, display device 1205may be integrated in system 1200 or implemented separately from system1200.

As shown in FIG. 12, and discussed above, the processing unit 1220 mayhave logic circuitry 1250 with an encoder 300 and/or a decoder 400. Theencoder 300 may have motion estimation unit 1252 and the decoder mayhave motion estimation unit 1254 to read and use the formula 1208 andprovide many of the functions described herein and as explained with theprocesses described herein.

As will be appreciated, the modules illustrated in FIG. 12 may include avariety of software and/or hardware modules and/or modules that may beimplemented via software or hardware or combinations thereof. Forexample, the modules may be implemented as software via processing units1220 or the modules may be implemented via a dedicated hardware portion.Furthermore, the shown memory stores 1204 may be shared memory forprocessing units 1220, for example, motion vector buffer 1206 and 1207as well as the ROM formula may be stored on any of the options mentionedabove, or may be stored on a combination of these options, or may bestored elsewhere. By one option, the neighbor block area MVs of thecolumns are stored in on-chip buffer 1206 while the one line buffer isstored in off-chip memory buffer 1207, and the ROM for storing theformula 1208 may be in either location. Also, system 1200 may beimplemented in a variety of ways. For example, system 1200 (excludingdisplay device 1205) may be implemented as a single chip or devicehaving a graphics processor, a quad-core central processing unit, and/ora memory controller input/output (I/O) module. In other examples, system1200 (again excluding display device 1205) may be implemented as achipset.

Processor(s) 1203 may include any suitable implementation including, forexample, microprocessor(s), multicore processors, application specificintegrated circuits, chip(s), chipsets, programmable logic devices,graphics cards, integrated graphics, general purpose graphics processingunit(s), or the like. In addition, memory stores 1204 may hold motionvector buffer 1206, and may be any type of memory such as volatilememory (e.g., Static Random Access Memory (SRAM), Dynamic Random AccessMemory (DRAM), etc.) or non-volatile memory (e.g., flash memory, etc.),and so forth. In a non-limiting example, memory stores 1204 also may beimplemented via cache memory. In various examples, system 1200 may beimplemented as a chipset or as a system on a chip.

Referring to FIG. 13, an example system 1300 in accordance with thepresent disclosure and various implementations, may be a media systemalthough system 1300 is not limited to this context. For example, system1300 may be incorporated into a personal computer (PC), laptop computer,ultra-laptop computer, tablet, touch pad, portable computer, handheldcomputer, palmtop computer, personal digital assistant (PDA), cellulartelephone, combination cellular telephone/PDA, television, smart device(e.g., smart phone, smart tablet or smart television), mobile internetdevice (MID), messaging device, data communication device, and so forth.

In various implementations, system 1300 includes a platform 1302communicatively coupled to a display 1320. Platform 1302 may receivecontent from a content device such as content services device(s) 1330 orcontent delivery device(s) 1340 or other similar content sources. Anavigation controller 1350 including one or more navigation features maybe used to interact with, for example, platform 1302 and/or display1320. Each of these components is described in greater detail below.

In various implementations, platform 1302 may include any combination ofa chipset 1305, processor 1310, memory 1312, storage 1314, graphicssubsystem 1315, applications 1316 and/or radio 1318 as well asantenna(s) 1313. Chipset 1305 may provide intercommunication amongprocessor 1310, memory 1312, storage 1314, graphics subsystem 1315,applications 1316 and/or radio 1318. For example, chipset 1305 mayinclude a storage adapter (not depicted) capable of providingintercommunication with storage 1314.

Processor 1310 may be implemented as a Complex Instruction Set Computer(CISC) or Reduced Instruction Set Computer (RISC) processors; x86instruction set compatible processors, multi-core, or any othermicroprocessor or central processing unit (CPU). In variousimplementations, processor 1310 may be dual-core processor(s), dual-coremobile processor(s), and so forth.

Memory 1312 may be implemented as a volatile memory device such as, butnot limited to, a Random Access Memory (RAM), Dynamic Random AccessMemory (DRAM), or Static RAM (SRAM).

Storage 1314 may be implemented as a non-volatile storage device suchas, but not limited to, a magnetic disk drive, optical disk drive, tapedrive, an internal storage device, an attached storage device, flashmemory, battery backed-up SDRAM (synchronous DRAM), and/or a networkaccessible storage device. In various implementations, storage 1314 mayinclude technology to increase the storage performance enhancedprotection for valuable digital media when multiple hard drives areincluded, for example.

Graphics subsystem 1315 may perform processing of images such as stillor video for display. Graphics subsystem 1315 may be a graphicsprocessing unit (GPU) or a visual processing unit (VPU), for example. Ananalog or digital interface may be used to communicatively couplegraphics subsystem 1315 and display 1320. For example, the interface maybe any of a High-Definition Multimedia Interface, Display Port, wirelessHDMI, and/or wireless HD compliant techniques. Graphics subsystem 1315may be integrated into processor 1310 or chipset 1305. In someimplementations, graphics subsystem 1315 may be a stand-alone cardcommunicatively coupled to chipset 1305.

The graphics and/or video processing techniques described herein may beimplemented in various hardware architectures. For example, graphicsand/or video functionality may be integrated within a chipset.Alternatively, a discrete graphics and/or video processor may be used.As still another implementation, the graphics and/or video functions maybe provided by a general purpose processor, including a multi-coreprocessor. In other implementations, the functions may be implemented ina consumer electronics device.

Radio 1318 may include one or more radios capable of transmitting andreceiving signals using various suitable wireless communicationstechniques. Such techniques may involve communications across one ormore wireless networks. Example wireless networks include (but are notlimited to) wireless local area networks (WLANs), wireless personal areanetworks (WPANs), wireless metropolitan area network (WMANs), cellularnetworks, and satellite networks. In communicating across such networks,radio 1318 may operate in accordance with one or more applicablestandards in any version.

In various implementations, display 1320 may include any television typemonitor or display. Display 1320 may include, for example, a computerdisplay screen, touch screen display, video monitor, television-likedevice, and/or a television. Display 1320 may be digital and/or analog.In various implementations, display 1320 may be a holographic display.Also, display 1320 may be a transparent surface that may receive avisual projection. Such projections may convey various forms ofinformation, images, and/or objects. For example, such projections maybe a visual overlay for a mobile augmented reality (MAR) application.Under the control of one or more software applications 1316, platform1302 may display user interface 1322 on display 1320.

In various implementations, content services device(s) 1330 may behosted by any national, international and/or independent service andthus accessible to platform 1302 via the Internet, for example. Contentservices device(s) 1330 may be coupled to platform 1302 and/or todisplay 1320. Platform 1302 and/or content services device(s) 1330 maybe coupled to a network 1360 to communicate (e.g., send and/or receive)media information to and from network 1360. Content delivery device(s)1340 also may be coupled to platform 1302 and/or to display 1320.

In various implementations, content services device(s) 1330 may includea cable television box, personal computer, network, telephone, Internetenabled devices or appliance capable of delivering digital informationand/or content, and any other similar device capable of unidirectionallyor bidirectionally communicating content between content providers andplatform 1302 and/display 1320, via network 1360 or directly. It will beappreciated that the content may be communicated unidirectionally and/orbidirectionally to and from any one of the components in system 1300 anda content provider via network 1360. Examples of content may include anymedia information including, for example, video, music, medical andgaming information, and so forth.

Content services device(s) 1330 may receive content such as cabletelevision programming including media information, digital information,and/or other content. Examples of content providers may include anycable or satellite television or radio or Internet content providers.The provided examples are not meant to limit implementations inaccordance with the present disclosure in any way.

In various implementations, platform 1302 may receive control signalsfrom navigation controller 1350 having one or more navigation features.The navigation features of controller 1350 may be used to interact withuser interface 1322, for example. In implementations, navigationcontroller 1350 may be a pointing device that may be a computer hardwarecomponent (specifically, a human interface device) that allows a user toinput spatial (e.g., continuous and multi-dimensional) data into acomputer. Many systems such as graphical user interfaces (GUI), andtelevisions and monitors allow the user to control and provide data tothe computer or television using physical gestures.

Movements of the navigation features of controller 1350 may bereplicated on a display (e.g., display 1320) by movements of a pointer,cursor, focus ring, or other visual indicators displayed on the display.For example, under the control of software applications 1316, thenavigation features located on navigation controller 1350 may be mappedto virtual navigation features displayed on user interface 1322, forexample. In implementations, controller 1350 may not be a separatecomponent but may be integrated into platform 1302 and/or display 1320.The present disclosure, however, is not limited to the elements or inthe context shown or described herein.

In various implementations, drivers (not shown) may include technologyto enable users to instantly turn on and off platform 1302 like atelevision with the touch of a button after initial boot-up, whenenabled, for example. Program logic may allow platform 1302 to streamcontent to media adaptors or other content services device(s) 1330 orcontent delivery device(s) 1340 even when the platform is turned “off”In addition, chipset 1305 may include hardware and/or software supportfor 7.1 surround sound audio and/or high definition (7.1) surround soundaudio, for example. Drivers may include a graphics driver for integratedgraphics platforms. In implementations, the graphics driver may comprisea peripheral component interconnect (PCI) Express graphics card.

In various implementations, any one or more of the components shown insystem 1300 may be integrated. For example, platform 1302 and contentservices device(s) 1330 may be integrated, or platform 1302 and contentdelivery device(s) 1340 may be integrated, or platform 1302, contentservices device(s) 1330, and content delivery device(s) 1340 may beintegrated, for example. In various implementations, platform 1302 anddisplay 1320 may be an integrated unit. Display 1320 and content servicedevice(s) 1330 may be integrated, or display 1320 and content deliverydevice(s) 1340 may be integrated, for example. These examples are notmeant to limit the present disclosure.

In various implementations, system 1300 may be implemented as a wirelesssystem, a wired system, or a combination of both. When implemented as awireless system, system 1300 may include components and interfacessuitable for communicating over a wireless shared media, such as one ormore antennas, transmitters, receivers, transceivers, amplifiers,filters, control logic, and so forth. An example of wireless sharedmedia may include portions of a wireless spectrum, such as the RFspectrum and so forth. When implemented as a wired system, system 1300may include components and interfaces suitable for communicating overwired communications media, such as input/output (I/O) adapters,physical connectors to connect the I/O adapter with a correspondingwired communications medium, a network interface card (NIC), disccontroller, video controller, audio controller, and the like. Examplesof wired communications media may include a wire, cable, metal leads,printed circuit board (PCB), backplane, switch fabric, semiconductormaterial, twisted-pair wire, co-axial cable, fiber optics, and so forth.

Platform 1302 may establish one or more logical or physical channels tocommunicate information. The information may include media informationand control information. Media information may refer to any datarepresenting content meant for a user. Examples of content may include,for example, data from a voice conversation, videoconference, streamingvideo, electronic mail (“email”) message, voice mail message,alphanumeric symbols, graphics, image, video, text and so forth. Datafrom a voice conversation may be, for example, speech information,silence periods, background noise, comfort noise, tones and so forth.Control information may refer to any data representing commands,instructions or control words meant for an automated system. Forexample, control information may be used to route media informationthrough a system, or instruct a node to process the media information ina predetermined manner. The implementations, however, are not limited tothe elements or in the context shown or described in FIG. 13.

As described above, system 1200 or 1300 may be implemented in varyingphysical styles or form factors. FIG. 14 illustrates implementations ofa small form factor device 1400 in which system 1200 or 1300 may beimplemented. In implementations, for example, device 1400 may beimplemented as a mobile computing device having wireless capabilities. Amobile computing device may refer to any device having a processingsystem and a mobile power source or supply, such as one or morebatteries, for example.

As described above, examples of a mobile computing device may include apersonal computer (PC), laptop computer, ultra-laptop computer, tablet,touch pad, portable computer, handheld computer, palmtop computer,personal digital assistant (PDA), cellular telephone, combinationcellular telephone/PDA, television, smart device (e.g., smart phone,smart tablet or smart television), mobile internet device (MID),messaging device, data communication device, and so forth.

Examples of a mobile computing device also may include computers thatare arranged to be worn by a person, such as a wrist computer, fingercomputer, ring computer, eyeglass computer, belt-clip computer, arm-bandcomputer, shoe computers, clothing computers, and other wearablecomputers. In various implementations, for example, a mobile computingdevice may be implemented as a smart phone capable of executing computerapplications, as well as voice communications and/or datacommunications. Although some implementations may be described with amobile computing device implemented as a smart phone by way of example,it may be appreciated that other implementations may be implementedusing other wireless mobile computing devices as well. Theimplementations are not limited in this context.

As shown in FIG. 14, device 1400 may include a housing 1402, a display1404, an input/output (I/O) device 1406, and an antenna 1408. Device1400 also may include navigation features 1412. Display 1404 may includeany suitable screen 1410 on a display unit for displaying informationappropriate for a mobile computing device. I/O device 1406 may includeany suitable I/O device for entering information into a mobile computingdevice. Examples for I/O device 1406 may include an alphanumerickeyboard, a numeric keypad, a touch pad, input keys, buttons, switches,rocker switches, microphones, speakers, voice recognition device andsoftware, and so forth. Information also may be entered into device 1400by way of microphone (not shown). Such information may be digitized by avoice recognition device (not shown). The implementations are notlimited in this context.

Various implementations may be implemented using hardware elements,software elements, or a combination of both. Examples of hardwareelements may include processors, microprocessors, circuits, circuitelements (e.g., transistors, resistors, capacitors, inductors, and soforth), integrated circuits, application specific integrated circuits(ASIC), programmable logic devices (PLD), digital signal processors(DSP), field programmable gate array (FPGA), logic gates, registers,semiconductor device, chips, microchips, chip sets, and so forth.Examples of software may include software components, programs,applications, computer programs, application programs, system programs,machine programs, operating system software, middleware, firmware,software modules, routines, subroutines, functions, methods, procedures,software interfaces, application program interfaces (API), instructionsets, computing code, computer code, code segments, computer codesegments, words, values, symbols, or any combination thereof.Determining whether an implementation is implemented using hardwareelements and/or software elements may vary in accordance with any numberof factors, such as desired computational rate, power levels, heattolerances, processing cycle budget, input data rates, output datarates, memory resources, data bus speeds and other design or performanceconstraints.

One or more aspects described above may be implemented by representativeinstructions stored on a machine-readable medium which representsvarious logic within the processor, which when read by a machine causesthe machine to fabricate logic to perform the techniques describedherein. Such representations, known as “IP cores” may be stored on atangible, machine readable medium and supplied to various customers ormanufacturing facilities to load into the fabrication machines thatactually make the logic or processor.

While certain features set forth herein have been described withreference to various implementations, this description is not intendedto be construed in a limiting sense. Hence, various modifications of theimplementations described herein, as well as other implementations,which are apparent to persons skilled in the art to which the presentdisclosure pertains are deemed to lie within the spirit and scope of thepresent disclosure.

The following examples pertain to additional implementations.

By one example, a computer-implemented method of motion estimation forvideo coding comprises receiving multiple frames of pixel data; anddetermining at least one motion vector candidate extending between acurrent block on a current frame and a reference block on a referenceframe, comprising: determining spatial neighbor block locations on thesame frame as the current block using a neighbor block pattern formulato compute at least one neighbor block coordinate relative to theposition of the current block on the current frame and usingsubstantially the same pattern formula regardless of the size of thecurrent block; and using motion vectors of the neighbor blocks atlocations determined by using the at least one neighbor block coordinateand to determine at least one motion vector of the current block.

By another implementation, the method may comprise using no less than8×8 block-based motion vectors from 8×8 neighbor blocks for currentblocks that have a sub-block size less than 8×8; wherein the block areapermitted to have neighbor blocks comprises at least one of: a singlerow above and adjacent the current block, at most three columns to theimmediate left of the current block, and a single row and a singlecolumn both adjacent the current block and having data stored by asingle line buffer of consecutive 8×8 pixel blocks the length of the topof the current block plus one block; the method comprising: bufferingonly 8×8 pixel block-based motion vectors as potential neighbor blockmotion vectors; determining at most nine spatial neighbor motionvectors; and determining at most two candidate motion vectors from themotion vectors to be used to determine the motion vector of the currentblock; wherein the neighbor block pattern formula using the width or theheight or both of the current block formula to at least one of:determine a row or column position of at least one neighbor block, aligna neighbor block at a corner of the current block, and align a neighborblock at a center of the width or height of the current block; whereinthe neighbor block pattern formula is provided at a decoder to be usedto determine neighbor block locations on frames at the decoder; themethod comprising: storing motion vectors of at least one column ofpotential neighbor block area on on-chip memory while storing motionvectors of a single row of potential neighbor block area in a linebuffer at other than on-chip memory; and indicating a block location isunavailable as a neighbor block to a current block when the neighborblock pattern formula places the neighbor block at a location outsideand to the right of a top area extending from a 64×64 pixel superblockthat contains the current block.

By yet another implementation, a computer-implemented system has adisplay, a memory; at least one processor communicatively coupled to thememory and display; and a motion estimation unit operated by the atleast one processor and being arranged to operate by: receiving multipleframes of pixel data; and determining at least one motion vectorcandidate extending between a current block on a current frame and areference block on a reference frame, comprising: determining spatialneighbor block locations on the same frame as the current block using aneighbor block pattern formula to compute at least one neighbor blockcoordinate relative to the current block and using substantially thesame pattern formula regardless of the size of the current block; andusing motion vectors of the neighbor blocks at determined neighbor blocklocations to determine at least one motion vector of the current block.

By another implementation, the system includes wherein the neighborblock pattern formula uses the width or the height or both of thecurrent block formula to determine a row or column position of at leastone neighbor block; wherein the motion estimation unit is operated byusing no less than 8×8 block-based motion vectors from 8×8 pixelneighbor blocks for current blocks that are a sub-block size less than8×8 pixel; wherein the block area permitted to have neighbor blockscomprises one of: a single row above the current block with motionvectors stored off-chip, and at most three columns to the immediate leftof the current block with motion vectors stored on-chip, and a singlerow and a single column both adjacent the current block; wherein eightneighbor blocks each are used for current block 8×16 or 16×8 pixel, andwherein at most nine neighbor blocks are used for other current blocksizes due, at least in part, to the neighbor block pattern formula;wherein neighbor blocks are eliminated due to duplication so that lessthan nine neighbor blocks listed by the neighbor block pattern formulaare provided for current blocks of different sizes and due, at least inpart, to equations provided by the neighbor block pattern formula; andwherein the neighbor block locations are determined without using alook-up table that lists fixed neighbor block locations depending oncurrent block size.

By one approach, at least one computer readable medium having storedthereon instructions that when executed cause a computing device tooperate by: receiving multiple frames of pixel data; and determining atleast one motion vector candidate extending between a current block on acurrent frame and a reference block on a reference frame, comprising:determining spatial neighbor block locations on the same frame as thecurrent block using a neighbor block pattern formula to compute at leastone neighbor block coordinate relative to the current block and usingsubstantially the same pattern formula regardless of the size of thecurrent block; and using motion vectors of the neighbor blocks atdetermined neighbor block locations to determine at least one motionvector of the current block.

The instructions cause the computing device to operate by: using no lessthan 8×8 block-based motion vectors from 8×8 neighbor blocks for currentblocks that have a sub-block size less than 8×8; wherein the block areapermitted to have neighbor blocks comprises at least one of: a singlerow above and adjacent the current block, at most three columns to theimmediate left of the current block, and a single row and a singlecolumn both adjacent the current block and having data stored by asingle line buffer of consecutive 8×8 pixel blocks the length of the topof the current block plus one block; wherein the instructions cause thecomputing device to operate by: buffering only 8×8 pixel block-basedmotion vectors as potential neighbor block motion vectors; determiningat most nine spatial neighbor motion vectors; and determining at mosttwo candidate motion vectors from the motion vectors to be used todetermine the motion vector of the current block; wherein the neighborblock pattern formula using the width or the height or both of thecurrent block formula to at least one of: determine a row or columnposition of at least one neighbor block, align a neighbor block at acorner of the current block, and align a neighbor block at a center ofthe width or height of the current block; wherein the neighbor blockpattern formula is provided at a decoder to be used to determineneighbor block locations on frames at the decoder; wherein theinstructions cause the computing device to operate by: storing motionvectors of at least one column of potential neighbor block area onon-chip memory while storing motion vectors of a single row of potentialneighbor block area in a line buffer at other than on-chip memory; andindicating a block location is unavailable as a neighbor block to acurrent block when the neighbor block pattern formula places theneighbor block at a location outside and to the right of a top areaextending from a 64×64 pixel superblock that contains the current block.

In another example, at least one machine readable medium may include aplurality of instructions that in response to being executed on acomputing device, cause the computing device to perform the methodaccording to any one of the above examples.

In yet another example, an apparatus may include means for performingthe methods according to any one of the above examples.

The above examples may include specific combination of features.However, the above examples are not limited in this regard and, invarious implementations, the above examples may include undertaking onlya subset of such features, undertaking a different order of suchfeatures, undertaking a different combination of such features, and/orundertaking additional features than those features explicitly listed.For example, all features described with respect to the example methodsmay be implemented with respect to the example apparatus, the examplesystems, and/or the example articles, and vice versa.

1. (canceled)
 2. At least one non-transitory article comprising acomputer-readable medium having instructions stored thereon that whenexecuted cause a computing device to operate by: receiving, by at leastone processor, multiple frames of pixel data; and determining, by atleast one processor, at least one inter-prediction motion vectorcandidate of a current block on a current frame comprising: determiningat least one neighbor block location on the current frame and includingat least one neighbor block adjacent the current block, wherein thecurrent block is not limited to being the same size as the neighborblocks, computing at least one neighbor block coordinate relative to theposition of the current block on the current frame and using asubstantially same neighbor block pattern formula regardless of the sizeof the current block relative to the neighbor blocks, and using motionvectors of the at least one neighbor blocks on at least one locationdetermined by using the at least one neighbor block coordinate.
 3. Thearticle of claim 2 comprising using no less than 8×8 block-based motionvectors from 8×8 neighbor blocks for current blocks that have asub-block size less than 8×8.
 4. The article of claim 2 wherein theblock area permitted to have neighbor blocks above the current blockcomprises a single row above and adjacent the current block.
 5. Thearticle of claim 4 wherein the block area permitted to have neighborblocks is at most three columns to the immediate left of the currentblock.
 6. The article of claim 2 wherein the block area permitted tohave neighbor blocks comprises a single row and a single column bothadjacent the current block and having data stored by a single linebuffer of consecutive 8×8 pixel blocks along the length of the top ofthe current block plus one block.
 7. The article of claim 2 wherein theinstructions cause the computing device to operate by buffering only 8×8pixel block-based motion vectors as potential neighbor block motionvectors.
 8. The article of claim 2 wherein the instructions cause thecomputing device to operate by: determining at most nine spatialneighbor motion vectors; and determining at most two candidate motionvectors from the motion vectors to be used to determine the motionvector of the current block.
 9. The article of claim 2 wherein theneighbor block pattern formula uses the width or the height or both ofthe current block to at least one of: determine a row or column positionof at least one neighbor block, align a neighbor block at a corner ofthe current block, and align a neighbor block at a center of the widthor height of the current block.
 10. The article of claim 2 wherein theneighbor block pattern formula is provided at a decoder to be used todetermine neighbor block locations on frames at the decoder.
 11. Adevice comprising: a display; a memory; and at least one processorcommunicatively coupled to the memory and display, the at least oneprocessor being arranged to operate by: receiving, by at least oneprocessor, multiple frames of pixel data; and determining, by at leastone processor, at least one inter-prediction motion vector candidate ofa current block on a current frame comprising: determining at least oneneighbor block location on the current frame and including at least oneneighbor block adjacent the current block, wherein the current block isnot limited to being the same size as the neighbor blocks, computing atleast one neighbor block coordinate relative to the position of thecurrent block on the current frame and using a substantially sameneighbor block pattern formula regardless of the size of the currentblock relative to the neighbor blocks, and using motion vectors of theat least one neighbor blocks on at least one location determined byusing the at least one neighbor block coordinate.
 12. The device ofclaim 11 wherein the neighbor block pattern formula uses the width orthe height or both of the current block to determine a row or columnposition of at least one neighbor block.
 13. The device of claim 11wherein the instructions cause the computing device to operate bystoring motion vectors of 1 to 3 columns of potential neighbor blockarea on on-chip memory while storing motion vectors of a single row ofpotential neighbor block area in a line buffer at other than on-chipmemory.
 14. The device of claim 11 wherein the instructions cause thecomputing device to operate by indicating a block location isunavailable as a neighbor block to a current block when the neighborblock pattern formula places the neighbor block at a location outsideand to the right of a top area extending from a 64×64 pixel superblockthat contains the current block.
 15. The device of claim 11 wherein theformula uses positions of the neighbor blocks relative to one or morecorners or proportional position along one or more sides of the currentblock.
 16. The device of claim 11 wherein the motion estimation unit isoperated by using no less than 8×8 block-based motion vectors from 8×8pixel neighbor blocks for current blocks that are a sub-block size lessthan 8×8 pixel.
 17. The device of claim 11 wherein the block areapermitted to have neighbor blocks comprises a single row above thecurrent block with motion vectors stored off-chip, and at most threecolumns to the immediate left of the current block with motion vectorsstored on-chip.
 18. The device of claim 11 wherein the block areapermitted to have neighbor blocks comprises a single row and a singlecolumn both adjacent the current block.
 19. A computer-implementedmethod of video coding comprising: receiving, by at least one processor,multiple frames of pixel data; and determining, by at least oneprocessor, at least one inter-prediction motion vector candidate of acurrent block on a current frame comprising: computing at least oneneighbor block coordinate relative to the position of the current blockon the current frame and using a substantially same neighbor blockpattern formula regardless of the size of the current block relative tothe neighbor blocks, and using motion vectors of the at least oneneighbor blocks from at least one location determined by using the atleast one neighbor block coordinate.
 20. The method of claim 19 whereinneighbor blocks are eliminated due to duplication so that less than nineneighbor blocks listed by the neighbor block pattern formula areprovided for current blocks of different sizes and due, at least inpart, to equations provided by the neighbor block pattern formula. 21.The method of claim 19 wherein neighbor block locations are determinedwithout using a look-up table that lists fixed neighbor block locationsdepending on current block size.
 22. The method of claim 19 wherein theneighbor block pattern formula is provided at a decoder to be used todetermine neighbor block locations on frames at the decoder.
 23. Themethod of claim 19 wherein the neighbor block pattern formula uses thewidth or the height or both of the current block formula to at least oneof: determine a row or column position of at least one neighbor block,align a neighbor block at a corner of the current block, and align aneighbor block at a center of the width or height of the current block.24. The method of claim 19 wherein the formula uses positions of theneighbor blocks relative to one or more corners or proportional positionalong one or more sides of the current block.
 25. The method of claim 19wherein the formula uses positions of the neighbor blocks relative toone or more corners or proportional position along one or more sides ofthe current block regardless of the size of the current block relativeto the neighbor blocks.
 26. The method of claim 19 wherein the blockarea permitted to have neighbor blocks comprises a single row above thecurrent block with motion vectors stored off-chip, and at most threecolumns to the immediate left of the current block with motion vectorsstored on-chip.